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Preface ' ^ 

This* paper is based on a presentation by the authors at the 1981 
Annual convention W the National Association of School Psychologists 
in. Houston, Texas. The data -reported represent the majbr findings 

of a number of IRLD research studies; the relevant research reports 

( 

are listed in Appendix A. A summary of the findings, which was dis- 
tributed at the presentation, is provided j'n Appendix B. 
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Qurrent Research on Psychoeducational Assessment and 
% Decision Making: Implications for Training and Practice 

4 ^ 

The University of Minnesota Institute for Research on Learning 
Disabilities (IRLD) is one of five institutes under federal contract « 
to conduct research in the area of learning d>sabi li ti6s ^The pri- 
mary focus of the Minnesota Institute is on improving the assessment 
and decision^makiftg' process . One goal of IRLD research has been to 
address the need for. documentation of current practices. Toward this 
end, a number of studies examjning the state of the art i r%educati onal 
assessment and decision making have- been conducted. The studies utilize 

various methodologies, including: 

* 

• Simulation of the decision-making process 

• Psychometric assessment of students 

• Surveys 

• Videotapes of placement teams • ^ 

• Longitudinal case studies 

• Reviews of student records 

Throughout the studies on assessment and decision-making practices 
in schools, we have been aware of the complexity of the process and the 

4 

many factors that influence educational decisions. Furthermore , we 

realize that there are no easy answers to the complex problems that 

affect these practices, and we propose no simple solutions. Our |^e- 

search efforts are an attempt to describe the process as it now exists, 

with the goal of provijding information on the characteristic? and ef- 

fectiveness of current practices that can lead to implications for 

1 • • 

"3 



improving services? to students. ^ 

This paper summarizes findings in two general areas: (a) informa-_ 
tion on contemporary^assessment practice3 gathered from individual 
assesso^s, decisioh makers , or students; and (b)^ i nformation gathered 
on the team decision-making process. Following these two sections, 
the impl^ications of the findings for school psychology are discussed. 
Reactions to the findings and implications then are provided by a 
practicing school psychologist (Betty Yanowitz) and by- an administrator 



at the state department level (John Taylor). 



r 



Contemporary Assessment Practices 
Indivjdual ^upil assessment has long been a major part of the 
school psychologist's activities^and an important aspect of the educa-' 
tional decision-making process. In regard to learning disabi 1 i ti^es ' 
specifically, we were interested in finding out what assessment devices 
are being used/ whether the'devices used are technically adequate, and 
how well they, ia fact, identify learning disabled students. We also 
were interested in identifying key factors that influence decisions 
abou"tS<udents. 

Six methodologies simula1;ion, assessment, sJrveys., videotapes, 
case studies, record reviews were used to explore these questions. 
Since results generally were consistent across studies employing the var- 
ious methodologies, two'major studies will be used tp illustrate key 
findifigs. Occasional mention will be made of other studies when appro- 
Rriate. The two major studies discussed here are; (a) a simulation 
of the decision-mikiag process, and (b) psychometric assessment of,LD^ 
and non-LD 1 ow-S^hieving students by IRLD staff. Jhe methodologi*es for 
these two studies are described Ipriefly to provide a context for dis- 
cussion of the findings. 

The^subjects for the computer-aided simul ation^ study incliided 223 
educational personnel from the Minneapolis/St. Paul metr*opol itan area. 
Subjects were schooT psychologists, special "^education teachers, regular 
education teachers, admini-strators , and support' personnel who had par- 
^icipated in at least two placement team meetings in their home schools. 

r 

A dia-^ram of the simulation process is presented in Figure 1. Each 



subject was administered a 25-item pretest assessing* knowledge of basic ^ 
measurement and assessment procedure^. After this w^s completed, the 
subject received a referral sheet for a hypothetical fifth grade^^tuderit ; 
the referral sheet Included such information as the child's name, age-, 
pare)its' occupaj:1on (reflecting Socioeconomic status SES) , a picture, 
of the child, and a statement of the referral problem. The referral 
statement {see Table 1) was geperal In nature and reflected either aca- 
demic or behavioral concerns. The sex, SES , attractiveness of the 
stirderit, and typt of referral problem were systematically varied to 
produce 16 conditions; each subject was assigned to one of these condl- 
tlons. . " . ^ ' 



Insert Figure 1 and Tabl^ 1 about here 



After reviewing the referral i nfornratlon, subjects were given access 
to^ technical , quantitative, anfl qualitative Informatlor^ on 49 commonly 
used assessment devices/procedures via a Telray remote computer terminal. 
Subjects could look at information on as many ^devices as they wished 
within a 25-m1nute time frame. All quantltati ve and qualitative assess r 
ment data provided 'were within the average, range for a fifth grade ^tu - 
dent . 

When the subject indicated she/he had finished reviewing assessment 

Information^ a series of decision questions was asked relating to eli- 

gibility for se^v1ces, clasf-1 ficafion, prognosis, and placement. .Subjects 

V 

also were ask&d to indicate the degree to whi^h the di fferenf types of 
information available had Influenced their decisions. 



In the second major study, 5,0 LD and 49 low-achieving fourth grade^ 
students were administered a battery of psychometric measures by IRLD 
staf^f members. . The LD students were, those Students identified by thei/ 
schools as LD within six months prior tto participation in tjhe study. 
The low-achieving'group consisted of those students who had not been ' 
identified as learning disab>ed by their schools b>ut'wh-o had scored at 
or below. the *25th percentile' in reading or math on the Iowa test of Bas'^c 
Skills and who showed aVerage or above average ability on school -admi ni s- 
tered group tests of general cognitive ability. The devices administered 
to these students are listexl in Table 2. This battery was constructed 
to include a representative sample of those devices commonly used to 
identify LD students. 

Insert Table 2 "about here^ ^ 

, f--- 

) \ ^ 

Resepirch findings front the two- studies djf^cribed above, as well as 

from other studies, were used to answer foui** key ques.tioris: 

f Wliat assessment devices are being used in maki^ng |denti- 

fixation, placement, and programming decisions? - 

• How technically a^dequate are the assessment devices that 
are commonly used? ^ \ ^ 

• ,How well do commonly used devices identify Isijirning 

disabled students? 

% What factors affect'the decisions b^ing made? 

WHAT ASSESSMENT DEVICES ARE USED IN MAKIN'g' IDENTI FI CATION , PLACEMENT , . 
AND PROGRAMMING DECISIONS ? 

In a study' in wKich" we' reviewed tha^school records of past and 
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prese/it students at a private day S'chool for, LD stud'ents, we found a 
large number of assessmervt* devices being us^d. The school itself had 
1^6 assessment devices -on fi le. ^ Of these, 42 w^>fi mentioned in the 
records of one or more of the .students , with 15 of these devices men- 
tioned fairly frequently. For any one student, jinformation ff^om 9 tO' 
18 devices was Recorded in the studejit's fi*le*. 

We also syrveyed 39 Child Seryice Demonstration Centers (CSDCs) 
about the devic^they commonly use to assess their students. The num- 
ber of devices mentioned by these model programs for LD students varied 
greatly, ranging from 3 to 39 (7 = 11.5, SD ^'^.3). Thvrty-^^^^^^ 
cflmmercial devices and^a variety of informal/center-devel o;)ed insttnWents 
were used by three or more centers. No one device was Jsed by all centers 
and only five devices were used by more than half of the responding" cen- 
^s . These five* devices were^ 

• Key Math Diagnostic Arithmetic Test ^ 

IT • ' • ' 

• Peabody Individual Achievement Test 

t Wechsler Intelligence Scale for Children (original or Revised) 

• , Wide Ra/ige Achievement Test- 

• Informal' or Center-developed tests- ^ 

In the simulation stud^, subjects coulV choose to look at infoi^^- 
tion on as many of th6 49 devices listed a/ they ^wished within a 25- ^ 
minute time frame. Of the 159 subjects'^whose choices were recorded, a 
total of 1014, devices were selected {I = 6.4 devices per subject) . In 
contrast to the emphasis on achievement-oriented devices by the CSDCs, 
none of the five most commonly selected devices in the simulation study 
were achievement measures. In fact, the most commonly selected , 
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'achievement measure, the Peabody Individual Achievement Test, was only 
the eighth most frequently selected device. The top five devices in 
the simulatton study were: • ^ 

• SWechsler Intelligence Scale for Children - Revised 

• "'^Bendei^iM^ Gestalt Test 

• ^ Peterson-Quay Behavioral Checklist 

• frequency counts ' , ^ 

• Stanfordj^B^j^t Intell igence Scale 

HOW TECHNICALLY ADEQUATE ARE THE COMMONLY USED ASSESSMENT DEVICES ? 

TechnicaTadequacy of assessment devices selected by subjects in 
IRLD studies was judged on the basi's of the criteria for norms, validity, 
and reliability set forth 1n APA Standards^ ( 1 974) . The various assessment 
.devices were considered tethnically adequate or technically inadequate 
with regard to each of the thrte characteristics in relation' to usage^ 
with rvormal populations; technical adequacy with regard to the use of 
_^a $sessment devices for an LD population rarely has been investigated. 
jI^Bfs should be kept in mind when coVisidering the following data. 

The percentages of technit^illy adequate devices selected in the 
first .through fourth selection dbring the simulation study are listed 
In Table 3. Approximately' three-quarter^ {77%) of the first devices 
selecte4 were technically adequate with regard to norms. Somewhat'^ more 
than half were technically ade^ate with regard to reliability and vali- 



dity (53% and 55X, respectively).' By the fourth selection, very few of 
the devices were technically adeqj^ate in terms of norms, reliability, or 
validity.^ The higher^^^te of technically adequate devices selected early 
in the assessment process may be explained partly by the use of a few 
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very commonly used devices that do evidence technical adequacy, such 
as the WISC-R. The pattern of decreasing technical adequacy'as the 
se2e£ii^ ^process continues alsf was evidenced in a survey of decision - 
makers in an eastern state and in information coOected frofn the Child 
Service Demonstration Centers* 

Insert Table 3 about here 
\ 

mA WELL DO COMMONLY USED 'ASSESSMENT DEVICES IDENTIFY LEARNING OISABLED ^ 
STUDENTS? 

In our review of records at a private school for LD students, it 
was noted that the assessment information reflected overal 1 underjichieve 
ment, with very few students scoring above the standardization mean in 
any area. ^ Also, of the 54 students for whom there were data available 
from both a standardized individual ability test and a standardised 
individual achievement test, over two-thirds had an ability-achievement 
discrepancy of less than one standard devi|||tion. ^ 

In the L&/I0W achievers' study, no differences of practical utility 
were found between the two groups when the scores on the psychome tries 
administered by the IRLD staff were examined. While tjie LD students, 
as a group, had scores significantly lower than the low-achieving group, 
on some measures , from 82 to 100 percent of the sUjdents in the^two 
gifoups earned scores within a common range on the 49 different subtest ^ 
and total test scoro^ Therefore, it >As not possible to tell whether 
a student was LD or low achieving on the basNs -of the test scores. For 
an example of standard*score distributions for LD and low-achieving stu- 
dents , see Figure 2 . 

Insert Figure 2 about here 



To'^test the possibility that patterits of test scores provide 
d^sion makers with the information needed to determine whether a 
child is classified as LD, set's of scores from eight non-LD and nine . 
LD students from the LD/low achievers study were, given to three groups 
of judges. One group consisted of 38 special education teachers, the 
second group was 65 school psychologists, and the third group was made 
up of 21 non-educators (university business and engineering students). 
Tfl^jXid^ classified each of the 17 students as learning disabled or 
not learning disabled on the basis of test scores. These classification 
'decisions then were compared to how each- student had been classified 
by his/her school and how he/she would be classifie(l using a 1 .C or 1 .5 
standard deviation ability-achievement discrepancy definition. Of the « 
decisions made by the 124 judges for the 17 students, approximately haVf 
matched the identification of the student as LD or non-LD as determined 
by the school's definition or the abil ity-achievement'discrepancy defini 
(s%e Table 4). The non-educators were just as accurate as the 
'^educators in determining which students were school-classified LD, or^ 
LD by the 1.0 standard deviation ability-achievement discrepancy defini- 
'tion. -'Further, the non-educators were considerably more accurate than 
the" educators when a more severe (1.5 standard deviation) abilUy- 
achievement* discrepancy definition was applied to the student's scores. 
In all other cases, the overall "hit rate" for any group of judges 
\4as approximately what would be expected if the students were randomly, 
classifigd. 

Insert Tab-le 4 about here 
# 

In the simulation study, 515; of the subjects indicated that the 




student was likely or very likely to be eligible for special education 

services. This occurred despite the fact that all of the assessment 

data. tl>ey reviewed reflected performance in the average range. The 

percentages of subjects within each professional role who indicated 

that the student Was likely to be eligible for services are listed in 

Table ^. Regu lar, teachers were most likely to declare the student 

eligible fo^ special'^.education services', while special educators were 

the Te^st likely to say the student was eligible for services. ^Yet, 

even 22. ]% of the special educators were willing to declare the student 

eligible, despite the student ' s average performance during assessment. 
* ^ _ ' 

Insert Table 5 about here 

The percentages of ^imulatio.n study subjects within each role who 
indicated that the student was likely or very likely to be mentally 
retarvjed, learning disabled, and/or emotionally disturbed are listed' 
in TabVr^. Overall, 62% of the sublets indicated that the student 
was likely to fall into one or more of the categories; the most frequently 
used'was learning -disabll ities . In fact, half of the s'pecial -education , 
teachers and school psychologists indicated that the sXudent was likely 
to bevlearning disabled. 

Insert Table 5 about here 

Only 16 of the 223 decision makers in, the simulation study clearly 
did not classify the student and als^ said the student was not likely 
to be eligible for special education servi-ces,. Recall that all of the 
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assessment data were wfthin the" average range; yet, only seven percent 
of the subjects indicated t'hat 'the student was not he^ndicapp^d . 
WHAT FACTORS AFFECT THE DECISIONS B€ING MADE? ' ^ ' 

Throug>iout the various studies conducted, decision makei^s indicated 
reliance on- intell igence and achievement data, the discrepancy between 
these two types of information, teacher input, and the statement of 
the referral problem as the bases for decisions. The reliance on a sub^ 
jective and vague referral statement was particularly evident in the 
Stimulation study. Not only did the subjects say that the referral state- 
ment influenced their decisions, but statistical analyses confirmed this, 
r example, even though the assessment data available were identical fbr 

al 1 slsiden^^those who had referral .statements focusing on behavioral 

/ 

concern^ wece more liiji^ly to be considered emotionally disturbed than 
those students 'whose referrals were more academically oriented. Con- 
versely, students with referral statements focusing on academic concerns , 
were generally perceived to be 1 earni ng .disabl ed . 

In^surthary, there Appear to be major problems with t-he assessment 
process as It currently exists. We found that decision makers rely on 
• 'many technically inadequate te$ts to make important educational decisions 
We also have found that regardless of the technical adequacy of the 
assessment devices used for identification, it is extremely difficult 
to differentiate learning disabled students from their low-achieving. ^ 
counterparts or even from average students. Of all" the information . 
available to decision makers, the original referral statement s^emj 
particularly inflifential in the decisions made. It is not yet clear 
whether this is due to the actual content of the referral statement, or 
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due simply to the fact that a referral was made and the search for a ^ 
psy<:hometric basis for the referral '^t in^tion. 

It is important to note that the results reviewed above were based 
on information gathered from individual .decision makers. While the 
input of individuals is an influential part of the decisiori-making process, 
actual eligibility and placement decisions now are made by mul tid-isci pi 1- 
nary teams of school' personnel. Therefore, a^ny investigation of the 

psychofeducat^ onal decision-liaking pr^ocess must consider the functioning 

\ ' * ' 

of the mul tidiscipl inary team. 
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Coatemporary Team Decision-Making Practices^ 

Another major line of research has been the investigation of team 

decision-making 'practices . Although multi-disciplinary teams are 

mandated by PL 94-1 42, there is little empirical research evidence on 

current practices and on the effectiveness of decision-maKi ng teams. 

One goal of IRLO' s Vesearch on team decision making has been to address 

, * ( 

this need fo^ docume'ntajion of current ^practices . 

There have been several IRLD investigations on team decision 

making; these have employed the following methodologies: 

r • ' . 

i 

^ ^# naturalistic observation 

f videotaping 

• longitudinal case studies ' . \ 

> 

• surveys 

One major study involved naturalistic observation and/or videotaping of 
38 pupil placement teains in Minnesota^ Observation and sti/dy of video- 
tapes of these teams were used to address sev«ra*l questions on current 
team decision-making practices. A second major investigation used a 
longitudinal procedure to follow seven^tudent casgs from the initial 
(Joint of.r'eferral to the final outcome .(e.g. , placement in special educa 
tion-). In order to gather information for the case studies, research „ 
investigators studied relevant documents , ^i nterviewed parents and pro- 
fess1(3it*4s involved with the case, and-attended child study meetings. 
Surveys also have been employed to gather data on team decis ma k Wg . 
Members of the 38 teams videotaped and/or observed in the naturalistic 
investigation at\^^red survey questions following the meetings in which 
they participated. Members of 23 decision-making teams .in Nebraska ^ 



also completed the same survey, but w£re not observed. Additionally, 
surveys of current team practices'were completed by 100 directors of 
special education 'irv,49 statej _and by 39 directors of Child Service 
Demonstration Centers. * 

' ^The following discussion combines and Summarizes findings across 
the various investigations in order to ^ddrass six major research questions 

• Wh^t are tKe characteristics of a "typical'V decision-making 
team? ^ ' 

To what extent do all team membersf^articipat.e in all aspects 
of 'the 'decision-making process? 

• What are the team members' perceptions of- the^ decision- 
making process? * 

• To what extent are parents active participants in the process? 

What are parents' views of the process? 
\ * - ' 

• What is the content of team meetings, ahd is there a relatij^n- 

V 

ship between th'e data presented and the decision reached? 

• To what extent 'do the teams meet criteria of effective team 
, functioning? _ . ' --^""^ • 

WHAT m, THE CHARACTERISTICS CPA ^TYPICAL" DECISION-MAKING TEAM? 

* Of the meetings- studied, the average team meeting consisted of 

t 

seven to nine professionals, lasted 'from 30 to 45 minutes, and considered 
/ ^ 

three options before placing a student. However, th^re was considerable 
variability and Inconsistency across decision-making teams. The, teams 
studied ranged in s.ize from 1 to 16 members, in length 'from 5 minutes 
to 2 hours, and considered 1 to 5 placement options.' Regarding team 
membership, fouc professionals were present most often at meetings; 



they were: a school administrator (usually a^ pri nc1 pal ) , a school 
psychologist, a speci-al education teacher, and a regular education 
teacher. 

TO WHAT EXTENT DO ALL TEAW MEMBERS PARTICIPA.TE IN. ALL ASPECTS OF THE 
DECISION-MAKING PROCESS? ^ 

For this question, we investigated both the degree of participation 
of the various professionals in the decision-making process and* the type 
of participation 'in the decision-maki ng "tasks of presenting data, offer- 
ing service options, and generating goal and method statements. The | 
most salient fin(Wng was that regular class teachers, who were among the 
top four profe^^iona.ls mo^t-^eit^n present at team meetings, were -not 
active participants in decision-maki ng functions. The majority of reguTar^ 
* class teachers.'^ participation in meetings donsisted of pre^^nting cJass- 
room and related data; less than 10% of regular class teachers' parti- 
cipation in meetings was spent in decision-making functions ^such as 
jnaking recommendations and proposing options. Regular class teachers 
were among the least active participants in prapos4ng service options 
for students. On the other hand, special education teachers, principal^, 
and school psychologis\s«were the most active participants in proposing 
options. Regular education teachers also ware among the least active 
in 1hitl6Lt1ng goal and method statements, even though they most likely 
have the greatest amount of educatiSKal information about the child and 
•would be responsible for intervening with the child. Relative parti- 
cipation by team members in the decision-marking tasks of proposing options 
and initiating goal and method statements are listed in Table 7. 



Insert Table 7 about here 
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. WHAT ARE TEAM MEMBERS' PERCEPTIONS OF THE DECrsiON-MAKING PROCESS? 

Team members generally reported they were satisfied with the out- 

come of the meeting and furthermore felt they were an important part 

< 

of th^ process. -Although a major purpose of mul tidisciplinary team 
meetings is to foster a sharing of information 'between participants, 
the majority of team members reported that their view of the child had 
Y ^ot changed as, a result of the meeting. In reporting factors that in- 
fluenced their decision, team members ^stated .they were influenced by 
data presented, not. by factors such as child characteristics, internal^ 
constraints (e.g., availability of services) or external pressures (e.g., 
advocacy groups) . The factors reported as most influencing decisions 
were teacher' reports of classroom achievement, information from the 

♦ V 

parent, and other sources of data (including observations, test scores, 
. ' etc.). (See Table 8.) ^^t is interesting -to note that information from 
teachers and parents is reported as being the most influential, yet 
teachers and parents are among the least active in participating in actual 
decision-making fujictions. 



Insert Table 8 about here 



To look at constraining variables in effectively Implementing 
team decision making, a national survey asked special education direc- 
tors to report the'xfaG<ors that operate as constraints in thei?^ dis- 
tricts. The factors most often reported were time and scheduling con- 
straints, lacl^ of adequate funds an4 ^taff, excessive paperwork, 
impediments to parent involvement, and the need for training in team 
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decision making. n 

TO WHAT EXTENT ARE PARENTS ACTJVE PARTICIPANTS IN THE DECISION-MAKING 
PROCESS? WHAT ARE PARENTS^ VIEW^ OF THE PROCESS ? 

Team members report^ that information from parents is a major in- 
fluence on their decisions, yet, in general, parents do not sew to be f 
active and i nformed^embers of the team decision-making process. Our 
findings regarding parent participation revealed that parents were 
never asked either their understanding of the meeting's purpose or their^ 
expectattonS for the meeting. Parents rarely were asked for their input 
in th6 meeting (and only then for verification of an observed problem), 
and in only 11% of the meetings was the Unguage at a level judged to 
be understandable to parents. Furthermore,- paVuots were among the least 
active participants" in proposing options and in suggesting goals and 
Oiethods, as can be seen^in Table 7. 

Another finding relative to pa(rents' views of the decision-mak'ing 
process was that, paradoxically, despite their lack of active involve- 
ment in decision-making for their children, parents generally had a 
positive view of the process. A typical' response* from parents was that 
they felt the school was concerned and 'doing Its best to help the child. 
However, questioning of parents a^o revealed that parents did not ap- 
pear to understand the decision-making process, the purpose of^the 
4iieetings, and the results presented at the meetings. Furthermore, 
pacents often may be disenfranchised of full partici pation j* n decision- 
making activities as evidenced by the finding that 79% of directors In 
the national survey reported that they hold meetings without parents 
present to come to a school consensus on a decision to be presented 
to parents. , 



/ 



18 

WHAT IS THE CONTENf OF TEAM MEETINGS. AND IS THERE A RELATrONS,HIP 
BETWEEN THE DATA PRESENTED AND THE DECISION REACHED? 

I The goal of this question was to describe the actual content and 
process of the meetings (i.e., to describe how time is spent in meetings, 
to describe what data are presented, and to document the actual outcome 
of the meetings). OverwhelTningly,'most of the time in 'team meetings Is 
spent in describing the child's problecn and in presenting data rather 
than generatirTg end discussing alternatives for. the child; Approximately 
30% of the meeting time observed was spent presenti/ig test datsh; 17% of 
the time was spent in discussing classroom performance. Anecdotal reports,- 
procedural matters, and tangential statements comprised the majority of 
theremai ning meeting time. A small proportion of this remaining meeting 
tim€ was spent in receiving parental input .and generating^nd discussing 
alternatives for programming. The rankings of various types of data dis- 
cussed if! meetings are listed In Table 9. 

/ 

^Insert Table 9) about here 

Furthermore, most of the data discussed in meetings, either non- 
test based or test based, were judged by researchers to be irrelevant to 
the final decision reached. We found essentially no relationship between 
the data, presented and the determination of eligibility for LD services 
accordl^ng to three major criteria. The correlations J/^^^een the data 
presented that supported each criterion and the decisions reached were 
calculated for the significant ability-achievement discrepancy criterion, 
the verbal -performance IQ discrepancy criterion, and the Federal definition 
criterion. In all instances, these correlations were low and non-signifi- 

4 

cant. (See Table 10.) However, the correlation between the amount of 



data presented and the decision reached was .52; in other words, 
the more data presented in a rrjeeting, the more likely the child. was 
to be labeled LB. • 

. ^ ' 

Insert Table 10 about here 

• TO WHAT EXTENT DO THE T^AMS MEET CRITERIA OF EFFECTIVE TEAM FUNCTIONING ? 

Criteria*^ for effective team functioning were formulated from a 
review of the literature and were used to rate the team meetings. These 
criteria are listed in Appendix. C and can be summarized in the major 
'categories of effective team functioning as follows: 

• The purpose of the meeti-ng is clearly stated., 

• The roles of team members are defined clearly and participation 
by all team members is encouraged. 

J The data presented are relevant to the decision to be made, 

• There 1s non-specialized participation by'all team mefbers 
(i.e., all members participate actively in-all aspects of 
the decision-making process). 

9 There is stru9tured separation of decision-making activities 
(i.e., there is clear delineation between -such activities 
as presenting information, proposing alternatives, arid evalu- 
ating options) . ' 
It was found that the teams studied did not meet' these basic cri- 
teria of effective team decision-making practices. Th'e overall findings 

were: * ^ 

• The purpose of the meeting sfeldom was stated explicitly, 

• The roles of team members hever were defined clearly and 
participation by all team members never was encouraged. 
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' . ^ • The data presented generally were not relevant to J:he final 

decision reached. 
• Team members did not partici pa^^equally in all aspects 
decision making; some team members clearly were more active 
in some functions than other team members. For instance, 
regular' class teach'ers rarely proposed options. 
; , ^ TheT»e was no clear delineation of activities; furthermore, in 

-^_ < ^88% of the meetings we could not ascertain the final decision 

„ made, or who m^^de it. , , . ' 

_ In summary, the major findings of IRLD research regarding team 

decision-making practices were: not all. team members (particularly 
regular classroom teachers and parents) actively participated in all 
aspects of decision making; most of the time in meetings was spent in 
, ' describing and discussing data that often were derived from techni^^ly 

' inadequate devices and often did not relate to the final decision reached; 
and finally, teams did not, meet the established criteria for Effective 
team functioning^^^^^^ 

From the re&earch on- current assessment and 'decision-making prac- 
tices, we have seen that there are ia joT probl ems . Action needs to be 
.taken to improve the effectiveness of assessment and team decision-making 
practices and to C|)nsider alternative? to current practices. Our findings 
^ ^ ^hmve nur^flbus^impiicatlons for Improv.ed practices, ^hese implications 

ig^r^ school psychologi-sts wlU^be discussed in the next section. 
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Research Impi ications 

Assessment and decision making reflect ^ complex interaction of 
the people making the decision, the data, and the characteristics of 
*the chit% within his/her social coj^text. There are no easy, simple 
solutions to problems 1n this complex process. There is no single 
way to assess; there is no single guideline for^decision making. 
Thereis no recipe for effective assessment and team decision making; 
at most, we have identified some necessary ingredient's. We would like 
to indicate at the outset that these ingredients require'ongo ing evalua- 
tion and should vary from district to'^district due to unique character- 
istics of each school systam. What makes sense for Grand Rapids, 
Michigan 'may not make^sense for Houston, Texas. ^ 

Implications for assessment and decision-making practices may be 
addressed most meaningfully by reflecting on the daily routine of school 
psychologists. The following scenario provides the context for under- 
standing the implications suggested by the synthesis of IRLD's research 
findings. 

The school psychologist receives a referral. Upon arriving 
at the referred student's school, the psychologist performs 
two tasks: 1) he/she administers a standard battery of tests 
including the Bender, WRAT, and yiSC-R, and 2) he/she speaks 
Briefly with the referring teacher. Other tedf?k members collect 
their data. The team meets, possibly several t)mes, to discuss 
and share Information regarding l>he referred student. Eligi- 
bility and placement decisions may be suggested or even dis- 
cussed. Next, the referral and assessment information is pre- 
sented to the parents, whose input is often limited to veri- 
fication of an observed problem. An lEP signature is obtained 
and the student is placed. Follow-up consultation may'^or may 
nbt occur. The school psychologist rece^ives another referral . 

Relate this scenario to the salient assessment and decision-making 

findfngs, which -include: 



• The reason for referral was very influential jn the 
decision-making process. 

• There was heavy emphasis on testing in the assessment/ 
decision-making process.^ ^ 

• The tests used were often technically inadequate. 

• When educators looked at psychometric data, their identifi- 
cation of learning disabled students was no better tha'n 

/chance.' 
• Psychometric di ffer/mtiation between low-achieving and 
V learning disabled students was not found. 

• , The more tests given, the more Vikely the child was to be 

called LD. ^ ^ . 

• Most of the time'in team meetings was spent in presenting 
data rather than generating alternatives for programming. 

• Parents and regular cl assroonv teachers participated the 
least in t^am decision making. 

• Data presented in team meetings generally did not relate to 
the final decision made. 

The impact of the referral statement was identified as extremely 
powerful throughout IRLD's research. We believe that the referral 
statement sets in motion the search for pathology - the search for 
a problem within the child - a search for confirming evidence, most 
likely with technically inadequate devices . Three questions come to 
mind: (1) when do teams know they have enough information to make 
an eligibility decision? (2) is the information solely test-oriented? 
(3) are team decision makers usi^vg relevant data, or, as suggested by 
Reynolds (19?5), are some students 'recei vi ng "more process than is due 
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The important issue for the practice of school psychology is how 
we, as school- psychologists, can meet the chal 1 enge of (mprovi ng assess-^ ^ 
ment and decision-making practices. The implications ofNshe research 
ff^ings include: the need for examination of underlying assumptions 
of current assessment practices, the importance of contact between the 
school psychologist and the referring teacher, the need for training* 
in team d^ision making to increase and equalizt participaticfn by^/e|rT! 
members, and the need for, on-going evaluation and documentation of .team 
decision-making practices. ' ^ 

Assumptions of Current Assessment Practices 

RecaU the first half of the scenario in which the school psycholp" 
^gist receives a referral, administers a standard battery of tests, and 
speaks with the classroom teacher. The research findi/igs suggest that 
it \% time for school psychologists^ to examine the underlying assumptj^s 
of current assessment practice. Do we really believe.it fs good assess- 
ment practice to administer routinely th« same^ests to all students 
referred? Do we really believe primarily test-oriented assessment re- 
flects good practice? Dd we really believe arbitrarily defined criteria^ 
such as ability-achievement discrepancies $ validly define learning disa- 
bilities? Rather than asking, ''What tests do I give?" perhaps we should be 
asking: (1) What decisions are we trying to make for the referred student? 
and (2) What behaviors- of the student must be sampled to make those decisions 

Decisions vary. Teams are asked to make eligibility, placement, 
instructional planning, a/id program eva/uation decisions. By identi- 
■fying the behaviors to l5e sampled to make different decisions, we will 
need to brbaden our view of assessment from primarily a psychometric • * 
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'orientation, an orientation that searches for presumed characterls.tics 
* within the child, to an orientation that emphasizes the environment 
in -which the student is expected to function. Standardized tests 
*are merely onie way to sample behaviors. Other tecftViiques include 
daily measurement of educational objectives, ^charting of interventions 
tried, examination of classroom expectations, teacher-made tests, 
teacher Observations, and structured (Observational systems measuring 
classroom characteristics such as instructional task, stud'ent response, 
•and teacher behavior. Assessment procedures should differ for a decir 
sion to place a student in a remedial math program versus the decision 
as'to whether a student has progressed in an i;itensive learning disability 
program. The challenge lies in our coll-fectiiig the appropriate data for 
the-decision to be rnade. It is challenging because it may involve crea- 
tivity and experimentation in sampling behaviors of the referred student. 
It may be that to make certain decisions, primarily classroom teachers 
or parents should be involved in sampling the behaviors. 

Second, regarding assessment practices^ we have a role to play in 
the overuse and misuse of tests. H^ye we fallen into the habit of 
administering a standard battery or do we ask, "Am I using the test for 
the purpose for which it wa5 designed?" Using a device technically 
adequate for screening purposes to assess' a student's yearly f}rogresi 
is inappropriate. The technical adequacy of devices must be. checked 
when using them for eligibility and placement decisions. 

By initially addressing the decisions to be made, we as school 
psychologists need to adopt the assumption that assessment is a process 
of problem solving, thereby differentiating it from tssting, which is 
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one set of methods for problem solving. Testing is % vial^le data 
collection method for certain decis"^ons; hpwever, utilization of al- 
ternative methods for problem solving may move us from-diagnostic pro- 
cedures that primarily serve to classify or label a problem within the 
student to diagnostic procedures that develop a set of working hypotti- 
eses about how best- to intervene with the student. As school psycholo- 
gists we have a responsibility to assure that children are not labeled 
solely on the basis of psychometric results. 

Given that psychometric informa'tic^has not proven successful 
in differentiating low achievers and the learning disabled, that 
the ability-achievement discrepancy is not a useful notion for instruc- 
tional planning, and that the goal of the law is placement in fhe 
'^east restrictive environment, we must broaden our view of assessment 
and train team members' in wdel assessment practices, including technical, 
adequacy of devices, approRriate data' col lection- methods , and emphasis, 
on intervention as opposed to placement. ^ 
School Psycholoqist-Seferrinq Teacher Contact J 

The school psychologist in the scenario spoke with the classroom 
teacher regarding the referral. This contact has important implications 
. for ass^sment and decision-making practices. The research findings 
Indicate that the manner in which the referral is made often determines 
the nature of the4ssessment prtcess and diagnosis. Referrals often 
are stated as conclusions and/or solutions to problems that h*ve already 
been identified by the referring teacher,. The research ' fi ndi ngs suggest 
that thereferral statement sets in motion a search for pathojogy - -a 
search foF^ presumed disability within the child. As a deterrent for 
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this search, the contact 'between the school psychologist and refer-* 

ring te^dier must focus on the firs> step in an assessment problem- 

solving orientation, that of problem clarification. 

Problem clarification includes identification of the problem 

* 

behavior, Comparison to peers /or a measure of deviancy, and examination 

'cJf instruotional/b^havioral interventions a^empt^d. General referral 

*^ * 

statements reflecting a diagnostic entity (e.g.,^th1s child has a ^ 
learning disability), specific methods of assessment ^(e. g. , I want IQ 
results), or an intervention (e.g., 1 want him placed in a^special class) 
^hould be replaced with statements of specific problem behaviors of the 
student (e.g., the student does not recognize letters; the student does 
not complete any written assignments in reading or language arts). If 
more than one problem behavior is identified, the teacher should rank 
order the severity of the^^^be^avlgrs * Next, 1t Is important to ask, 
"How -deviant 1s the problem b'^avlor?" Data should -be collected to 
compare the referred student with his/her peers. Finally, analysis of 
instructional a/id classroom interventions attempted 1s necessary. Con- 
sultation regar(fing a classroom intervention (either academid or be- 
havioral) may conceivably, although not necessarily, halt the assessment 
process. Consequently, these: steps will screen appropriate referrals 
to b6 considered for a more thorough assessment. . 

* School psychologtsts must 'begi'h to acknowledge that just because 
^a student is referred does not necessarily ?nean the student has problems 
warrant^^ .psychoeducatignal testing and/or a special education label. 
Rather, It may indicate a si tuation requi r1 ng teacher consultation for 

academic or 6ehav1or problems. As school psychologists, we must stop 
\ 



perpetuatiQg the automatic referpl -to-placement process .that is 
occurring for so many students. A more appropriate initial emphasis 
would be on the implementation of i nterventions , i n the classroom. 
This should serve to reduce time-consuming psychometric evaluations 
and improve instruction for children in the classroom. 
Team Decision-Making Practices 

Recall' the second half of the described scenario involving team mem- 
bers and presentation of the findings to parents. >f?hpl i cat ions of IRLD's 
research findings suggest that ongoing inselrvice training at various 
stages In the team decision-making process is needed. Inservice train- 
ing appears warranted to train teams initially In the use of criteria for 
effective teanr functioning, to examine the purpose of multidiscipl inary 
teams, and to Increase the participation of the regular classroom teacher 
and pareftits in the decisiorv-making process through explanation of assess- 
ment' fi ndi ngs and Instructional interventions. ''Each Inservice possibility 
will be addressed accordingly. 

The research findings suggest that the organization and structure 
of teams is Important for effiden^ decision makring. When special edu- 
cation directors werfe asked to name constraints in Implementing the 
team process, time and scheduling, lack of funds, and the need for train- 
ing for decision making were cited. School psychologists have a role to 
play in training teams for effective decision making. This training, 
which should emphasize organized procedures,' a clearly stated agenda, 
partici pdtipn by all members, and use of data relevant to the decision, 
should enable teams to be more time efficient and cost effective. 

Our fi^ndings raise two questions about the purpose of muUj- 
disciplinary teams. First, a major purpose of these teams Is to foster 



28 • . * 

a sharing of inTormation between participants; yet, the majority of 
team members reported that their perception^ of the, stu3ent had not 
changed as a result of the meeting. If a'purpose of the team meeting ^ 
is to .foster a'mul tidisciplinary vfew Of the student in which each team 
member adds a piece that contribu^d to the totil picture, we would 
expect members' views to be altered in meetings^ Since team members' 
views'were not significantly altered, it-does not appear that this goal 
of mul tidisciplinary teams was met. Second, the research, findings sug- 
gested that an examination of team members' 'beliefs regarding assessment 
and the team role is necessary. Te^s should ask, "Are we functioning 
as a team to d-iagnose and place or as a team to improve teaching, instruc 
tion, and learning?" Teams need to increase time spent in generating al- 
ternative service options and instructional interventions for the student 
It is time to refocus teafm thinking away f-rom who will serve the student 
to how can^we, as Jeducators , teach the student. Hopefully, such a re- 
focusing will lessen the number of cases in which a student goes through 
the lengthy team decision^klng process only to be declared ineligible 
and thus returned to the referring 'teacher who still does. not know what 
to do for- the student. , * - 

School psychologists have a role to play in staff training for the 
development of a mutual support system between regular and special 
education. Admittedly, Inservice training addressing the two issues 
raised is difficult because questions rather than answers appear awst 
evident. Perhaps.,the place to begin is to discuss such questions as: 
(1) Are team members /nerely presenting the data represented by their 
discipline (e.g., cognitive', achievement, observational), failing to 
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integrate it^across disciplines? (2) Are team, members knowledgeable 
regarding the data collection methods and inter||||^tat ion of each 
discipjine represented? (3) Are team members informed about current 
research and behavioral /academic*- interventionr for students? The con- 
sultation ski^Us of the school psychologist will be valuable in ad-' 
dressing and improving the goal of the mul tidi sci pi itiary team function- 
ing in each school . 

Non-spectalized participation is one of the criteria for effective 
decision making. However, ouf results indicated that while information 
provided by regular classroom teachers and parents was considered most 
valuable by all team members, parents and classroom teachers were among 
the least active participants in decisioa maki ng. School psycholog>i sts 
have training 1n group, process skills and need to utilize this training 
to 1ncrease..meani ngful participation by teachers ^nd parents. 

Regarduig the participation^^f the classro(Jm teachers, other ^ 
research (Bas'i i Leavitt, 1963) has suggested that for a decision to 
be likely to be implemented, the person who will be implementing the 
decision should take an active part in the decision-making jjrocess. 
Since regular education teachers are often involvetf in the day- to-day 

implementatio'p of team decisions, their lack of active paWicipatlon in 

■J 

decision making is disturbing. School psychologists have a role to play 
in inservicing regular classroom teachers in the principles of individual- 
izing instruction, alternative intervention strategies, and class' management 
techniques. Without tM s essential knowledge base, it is erroneous to ex- 

I 

pect classroom teachers to be active participants in the team decision- 
making process, particularly in setting goals and i'dentifying methods of 
instruction for students. ^ 
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Similarly, our finding that jargon was used 87% of the )/ime at the 
videotaped team meetings, suggests that school psychologists have a 
role to play in translating t*5e jargon into concrete observable behav- 
iors and in educating parents regarding. assessment findings and., educa- 
tional alternatives. It may be that parents "^o not actively participate 
In meetings because they are receiving technical information for the 
first time. Our results also indicated that teams often discussed a 

r 

3*g£nt over a- series of weeks before holding an eligibility conference. 
R^^gnizing that time is a precious resource, we suggest that the *teams 
we observed could have made the team process more efficient, increased 
participation by parents and classroom. teachers, and itifluervced instruc- 
tional planning by real 1 ocati/ig team time to include two preparation 

conferences. At the 5chopl preconference all school members would meet 

' ' ^ ' f 

once to share aRd^Tfftegrate assessment findings. At the parent precon- 
ference, the school psychologist could serve as the" team representative 
to convey the integrated assessment picture to the parents, to outline 
the purpose and agenda of the full team meeting, axid to encourage ques- 
tions. Additional reading materia-ls on assessment and Intervention 
techniques, preferably handouts written by team* members , could be pro- 
vided to the parents at this time.- The preconferences shpuld ensure 
understanding of the assessment information and crucial issues to be 
.addressed, thereby enabling a]2 participants to corranunlcate Ind bp 
actively involved in the decision, which should result in greater com- 
mitment in implementing instructional changes. 
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Evaluation and Documentation of the Team Process' 

Finally, the research findings offer a challenge to practicing 
school psychologists to conduct school-based research and evaluation 
of their districts' assessment and decision-making practices. Our 
results suggest some possible areas of investigation: longitudinal ♦ 
study of students involved in the decision-inaking process, evaluation . 
of specified instructional interventions, and'eval uatiort of inservice 
training and tearr; decision making. On-going evaluation and documentation 
of the team process is necessary and, as school psychologists, we should 
be engineering this. 
Final Comments 

^ We recognize that numerous and varied ifnpl ications for changes in 
the assessment and decision-making process Kave been addressed. In 
part, this is d^e to the comprehensiveness of the research findings, 
^n part, it is due to our feeling that there are no^ simple solutions 'to 
the problems in the complex process; there is no one recipe that. can be 
followed for assessment and decision making. ^ 

Furthermore, we want to emphasize tha^ we are advocating change; 
however, we are not suggesting elimination of all current' practi ces . 
Although we feel 'change must occur to broaden assessment techniques, 
we are not advocating elimination of testing. Although we feel assess- 
ment must begin with implementing instructional/behavioral Interventions, 
we are not advocating abolition of comprehensive psychoeducati onal diag- 
nostic studies. Although we feel inservice training or continuing edu- 
cation to broaden the knowledge base of aTI^ educators is necessary to 
improve instruction for aH students, we are not advocating the demiser 

3j- 
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o.f Special Education. 

The finding that the referral statement was Influential in the 
assessment and decision-makfing process - a findfc that suggests th^t 




if a student is referred there is a high pr^M|Hny tl^e student will 
be placed - allows us^ to advocate for yet aTTother change. We do not 
believe that all stents referred (or even a high percentage) .require 
placements.' It is our feeling that referrals require interventions 
at many levels, from instructional changes within the regular classroom 
to intensive self-contained special educatio/i programming. We advocate 
for a chan^ from referral-to-placement to referral -to-ifltervention . 

The important issue for school psychologists is to meet the chal- 
lenge of improving assessment and decision-makin^^practices within each 
school district. 
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Reaction 
by 

^ Betty Yanowitz 

School Psychologist 
Granite Schi^ol District, Salt Lake City, Utah 

' My first reaction upon hewing the results of the IRLD studies was, 
"Oh, no! That's mel I've been found out'." My sense of guilt, hovSever, 
""""w^ -quickly replaced by a feeling of re-lief as I thought of all the other 
"games" and "manipulations" I employ under the guise of decision making 
that have remained undiscovered and undisclosed in these otherwise com- 
prehensive IRLD studies. • 

Then I thought to myself,* "Why am I letting all the researchers make 
me feel guilty when everyone else I work with Is praising me. In spite of, 
or maybe because of, my often irrational decision-maktng practices?" 

The parents I work with, for example, respond almost cheerfully wtjpn 
I tell them of thelr^child's newly discovered "Learning Disability." Many 
are relieved to hear that some "expert" is finally agreeing that their 
child has a legitimate, social ly-a;:cepted problem. At last, they are not 
being labeled as "bad parents" who have neglected to provide a stimulating 
learning environment for their child. Ifi addition^ I am 6ffering parents 
a possible alternative to the frequently demanding and demeaning regular 
classroom teacher - a trained learning disability specialist who is certain 
to understand and help with the "problem." 

The students I work with are also happy when informed of their disa- 
bility. No longer are they*l)eing told the^re lazy, crazy, or both. 
In fact, the same assignment that was criticized on Jionday as being in- . 
complete and inadequate,^on Tuesday, is praised for Its effort and 
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achievement in light\}f the studept's newly labeled handicap. Competency 
tests are no longer an obstacle a^ students with identified problems are 
ge(Tefdly excusecj from' passing these threatening exams. 

The regular teachers in my school are also pleased with me .for ab- 
solving them of their responsibility for these non-'achieving students and 
for takinj.^^fl'ft^ identified problem child off their hands. My learning 
drsabi 1 ity' teachers are graetefuKto me for justifying their program and 
allowing them to t'feach remedial reading -to rather pleasant, berrign * chi 1 - 
dren. If one of these students dares to misbehave in the^ LD class, the 
teacher knows she merely has to yfel 1 "Behavior'a^ly'Hadadicapped," and the 
assessment and* label ing .proc^|^eg1 nsr amew.. 

Members of the Associat^llm' Parepts of Children with Learning Dis- 
abflities are pleased with me for identifying more potential menjbers to . 
march and demonstrate before the legislature in demand of more' funds to 
serve the needs of their handicapped chil-dren. Our State ^Office o,f Edu- 
cation, once 'assured that all the required signatures are in-their proper 



placesvJs happy that I am v^^kirrg in compliance with PL 94-142, thereby 
ensuring our state its al ienable share of federal, funds 

Thus, in spite of your discomforting research, my fellow practi- 
tioneVs.and I praise ourselves for our clever ways of "beating the system 
t6 'get help for needy kids. We rationalize thcit what we are doing is 
totally justifiable because a perusal of the literature reveaj/s that 
even the researchers can ' t agrele .oa what a learning disability is or 
how to measure this elusive handicap . A review of ^tate regulations 
reflects -such a variety of interpretations and criteria for evaluating 
learning disabilities that one must wonder if we are all describing th^ 
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same condition. Maybe the reason that we psychologists rely so heavily 
*on teacher referrals in decision making is that we* can frequently see a 
rationale and consistency to teachers' criteria not found elsewhere 
in our regulations or in our test kits. ^ . 

Yet,' in spite of these rationalizations, I arfTextremely concerned 
that, as practitioners, we are making a ser?ous error in buying into 
this irrational' system^under the guise of helping children. It is in- 
evitable that our statB legislators will become aware of studies such 
as. those conducted by the IRLD in Minnesota. In this budget cutting era, 
I worry that this information will provide a justi f i.cation .for eliminating 
or underfunding any or all special education programs necessary for help- 

ing kids. - - 

Furthermore, I believe that in perpetuating this system, we are 
cheating the children we believe to be "truly" "learning disabled; fn 
spite of my cynicism about &ur over-identification of LD children, I 
believe that such students actually do exist. (Of course, I must admit 
to frequently suppressing my impulse to take these children home and 
frame ttiem when I do find them.) By usjng LD. funding and services to 
help*so many, we are not' adequately serving the justifiable needs of the 
true learning disabled child. In addition, in playing these "games,"' 
we are neglecting our moral and ethixal responsibility for recognizing 
and changing a system we know is wrong and inequitable, ^ 

In conclusion, I would recommend that we, as practitioners, take 
a proactive stance and join with parent organizations, school adminis- 
trators, state offices of education, and 1 eijjislators to work on improving 

and amending the odious labeling system under which we work. We should^ 

I 

be concentrating on fixing the system rather than mendjng the child. 
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Psychologists must take an active leadership role before once again 

we are left behind, the perpetual victim, wondering what has happened 
to all our good intentions. . % 
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Reaction 
John Taylor 

Director of the Bureau of Psychological and Social Work Services 
Louisiana State Department of Education 
Office of Special Educational Services 

fortunately, when I reviewed the findings of the IRLD's research, 
I found that I was not surprised. In -fact, g^iven the numerous court cases 
in which testing practices were at least one of the issues, the plethora ' 
of research articles critical of the standardization and validation of 
cormnonly used instrumentation, and the constant controversy surrounding 
vari'gus" diagnostic categories ,• no one should be surprised. The, IRLD 
findirtgs are another clear ind^catiuDn that all is not welj in school 
psycholagy. The school psychology, behavior therapy, and psychometric 
literature' has for years pointed to the need for school psychology to 
abarrdon its over-reliance upon diagnostic testing as its primary service. 
However, q^ctual field practice has not, for many reasons, been able to 
ch^j^^ that rapidly. These resear'ch findi ngs 'hold certain implications 
for the field of school psychology at both the applied practice level and 



j:he administrative/state department levels. I would like to briefly note 
'scjne of^ these implications in the hopes that more rapid change may occur. 
Implications for the Field 

School psychologists clearly must be more selective 4n their choice, 
of instrumentation when conducting assessments of children. Unfortunately, 
this change Tn behavior will prove somewhat difficult for at least thi«€e 
reasons. First, State Department of. Education criteria for special education 
placement frequently demand extensive testing. As was noted in the IRLD 
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research, school psychologists generally do quite well in their selection 
of the first few instruments used, *but the quality of the instrumentation 
drops as the number of instruments used increases. As such, if a school 
psychologist uses several instruments in order to conduct a comprehensive 
evaluation of a suspected handicapped child, the school psychologist 
will, -In a.U likelihood, choose at least some poorly standardized ins^tru- 
ments. -Second, test publishers continue to market assessment instruments 
of established poor qual i ty ."^"^nally , university trainers continue to 
Instruct new school psychologists in the use of instruments which probably 
should not be used at all in any standardized fashion. 

School psychologists clearly need to establish that their "yalue to 
education goes far beyond their ability to admini-ster and score psychometric 
and educational tests. A recent article by ATtemose and Williamson (1981) 
suggests that in the near future the school psychologist may be replaced by 
a computer if the only activities that the school psychologist continues to 
engage in are those of administering and scoring tests. While Altemose and 
'Williamson maintain that the school psychologist is ^safe for a little while 
longer because computers cannot as yet administer the tests or exercise 
clinical judgment in the scoring and interpretation of the tests, they 
failed td mention that group tests can be administered by teachers and 
scored by computers with results in many cases similar to those obtained 
by a school psychologist in an individual administration oT the Jest. 

I do not believe that it would be difficult to develop a computer 
program ^hat uses teacher-administered group intelligence and achievement 
testis to generate a computed profile on the student's strengths and weak- 
nesses and an automatic decision as to whether or not the child is eligible 



for special education pTaceiDent; While thi.s is possible, I do not 
^believe that it is advisable. ^ believe the school psychologist can 
provide more valuable information and services to the school , such as 
teacher consultation, individual and group therapy with chi Idren,^ and 
direct interventions in the classroom. 

The results of the sirouktion study and the LD/Low Achiever s^dy 
revealed that if these practices are common in the field, a large number 
of children identi f ied^ as handicapped are in fact not handicapped. Some 
Of these errors in classification may' be attributed to faulty instrumen- 
tation, but the results of our own monitoring suggest that reliance upon 
vague referral statements and*an attitude that any child with significant 
school problem*5 needs special education are eqiwlly valid reasons for 
these errors. Our typical practice seems/ to be to assume that a referred 
child is handicapped .('iguilty".) and to assess to find the handicap instead' 
of assuming that the child i^ not^handi capped ("innocent") and assessing to 
discover the child's learning strengths and weaknesses and to make recommen- 
dations for i nstructi^on/treatment* It is this attitude, I believe, that 
leads professional assessment personnel to misclassify so many children 
as handicapped. The belffef that any child with learning proDlems will 
t^nefit from special education whether or not they are faaodicapped is 
just that, a belief. Sever^^l research articles suggest that this belief 
may be erroneous, and that children may in fact receive raore harm than 
benefit by being inappropriately^classi fled as handicapped when they 
are not. ' « 

Finally, there is an aspect of .the IRLD findings which were unmentioned 
in the report itself. Th«t is, i f ^assessment personnel frequently utilize 
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poorly standardized instruments, if faulty decision-making practices 
are engaged in by mul tidiscipl inary teams, and if children frequently 
are misc.1 assi fied, then one must wonder whether the recommendations 
for education and treatment'made by the^ assessment personnel are at 
all valid. Teachers and parents aVike frequently criticize the reports 
written by professional assessment personnel, mainlainipig that they 
are of little use in the instructional program, written in too much 
jargon, and often seeming to describe a child other than the one who 
wa^ referred. It may be that^this criticism is well deserved in light 
of our reliance upon the poor instruments and our frequent misclassifi- 
cations. 

Implications for Administration/State Department of Education 

Faulty evaluations may lead to inappropriate placements of c^vildren 
^in both special and regular education. As suc^, the findings of the IRLD 
studies have implications for the establ ishmefTt of teacher/pupj 1 ratios, 
class sizes, needed related services, and typrt of instructional programs 
utilized in special and regular education. This is so because state 
departments frequently follow trends in the identification of exceptional 
children and must make plans to have available sufficie^nt numbers of 
teachers and support personnel to serve these children.^' If the classifi- 
cations and racomnendatlons of tnese children are 1n error, then it may 
well be that the entire instructional and planning s'ystem in a state is 
in error. 

Similarly, tra1 ni ng programs in education and psychology follow 

trends. If more children are being identified as emotionally disturbed 

or learning disabled, then it is likely that more teachers will be trained 

*> • 

in the universities to serve these children, and that the emphasts m 
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professional ae^sessment courses will be on procedures for identifyiag 
learning di'sabled or emotionally disturbed children. I'believe this 
has- tiappened , and I am concerned that to some degree inappropriate 
identification practices have led to a "tail wagging the dog" phefiomena 
in professional* university training. - ^ ' 

Finally, rost state departjn^ts conduct regular monitoring activities. 
The IRLO findings clearly outline some of t>ie activities engaged in by 
professional assessment personnel in local school systems that should be 
closely reviewed during these monitoring visits. For^nstance, in our 
own state, we foupd that in one year the evaluations of 50%'of the children 
identified as handicapped did not m/et state approved criteria for eligi- 
bility. While many of the ^'compliance" issues were of a petty nature, 
an equal number had to do with such factors as choice of i nstr^umAitation , 
relationshi^p of the data gathered to the -final decision, and disregard of 
the cri teria^-findings very similar to those of-^the IRLD studies. ^ ^ 

. As. a- final note, these findings force one to reconsider the results 
of research studies conducted using children Identi fied -ds^andi^pped in 
public school settings^ Numerous studies have been conducted in which 
researchers 'attempt;ed to discover those salient factors that discriminated 
learning disabled children from regular c^)11dren, for instance. If as 
many children are misi denti f led 4s handicapped 1 n the public schools as 
these IRLD results suggest, then one Tnust be very skeptical of tne results 
of any research study that used already identified children as one popula- 
tion. In fact, even in the IRLD studies, questions^ must be raised to 
whether the* learning disability population was in fact really learning 
disabled. » 
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These findings are far reaching, -and of critical importance to the 
future of school psychology.' While many of my colleagues may be critical 
of this j^esearch for psychometric reasons (e.g., the restricted samples, 
small numbers of subjects), I find that these results are very much in 
keeping with the findings of our own state monitori;ig teams, /hey are 
both real' and significant, and demand a change in beljavior'on the part 
of school psychology, for if it does not change, it will very soon find 
•itself to be in little demand. 




43 

References 

» 

Altemose, J. R., & wmiamson, K. B. Clinical judgment vs the computei 
,Can the school psychologist be replaced by a machine-? Psychology 
in the Schools , 1981, 3, 356-363. 

^PA. Standards for educational and psychological tests . Washi)igton, 
D.C: American Psychological Association,^ 1 974. . j 

Bass, B. M., \ Leavitt, H. .J. Some experiments in planning and 
^ ' operating. Management^ Science , 1 963 , 2» 574-58S. 

Reynolds, M. More process than is due*- Theory into Practice , 1 975, 
14, 61-68. 



C 



j 



50 



r 



{ 



44 , . ^ 

Footnote 

Appreciation is extended to all individuals- and school districts 
'who participated in this research- Specia'l thanks are given to Linda ^ 
'Stevens for her help in preparing the original presentation, to Martha 
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Table 1 

Simulation Study Referral Statements 
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Type of Referral 



Statements 



Behavioral 



i 



AcatJemi c 



1. Belittles other children 

2. Talks back to adults 

3. Demonstrates temper tantrums 

4. Repeatedly fights with others 

5. Criticizes and nags others 

6. Annoys other children 

1. Fails to complete academic assign- 
ments in class 

2. Learns slowly 

3. ' Spells poorly 

4. Reads poorly 

5. Makes fal^ling grades in arittiffhetic 

6. Fails to complete homework 
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Table 2 

PsychometrKs -Administered in LD/Non-LD Comparison Study 



Wechsler Intelligence Scale for Children -.Revised 
Peabody Individual Achievement Test 
Stanford Achievement Test (selected subtests) 
Bender Vljsual Motor Gestalt Test 
Developmental Test of Visual -Motor Integration 
Piers-Harris Self-Concept Scale 
Peterson-Quay Behavior Problem Checklist 

Woodcock-Johnson Psycho-Educational Battery: Tests of Cognitive 
Ability and Tests of Achievement 
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Table 3 



^Percentages of Technically Adequate Devices 
Selected in Simulation ^tudy 









Norms 


Rel iabil i ty 


Validity 


Selection 1 






77 


58 


55 


Selection 2 






46 


57 


. 39' 


Selection 3 , 




' 33 ■ 


40 • 


. 26 " 


Selection 4 






17 


23 


15 
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Tame 4 

Judges' Accuracy in Identifying LD *an(^ -Non^LD Students 



Special^ Education School Non- 
Teachers \Psycho1ogists * Educators 

"School Definition 53% ^ '5S%^ P 55% 

Achievement Qeficit 55% 55!^. 51*. 

1 .5'SD Ability- 

Achievement Deficit 55% 56% ' 73% 




«5. 



0^ 



4^ 



' Table 5 

Numbers and Percentagestof Decision Makers in Simulation Study* 
Declaring Student Eligible for Special Services 



Regular Teacher 


t 58 


3 


61 


.2 


School Psychologist 


30 _ 




53 


.3 


Administrator 


23 
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4 • 


Special Educator 


84 • 




32 


1 


Support Personnel 


23 




52 


1 




5 -J 
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Table' 6 , 

Percentage of Decision Makers Classifying Normal Student 
as^Mentally Retarded (MR), Learning Disabled (LD), 
-and/or Emotionally Disturb'edH(ED) 



School P^chologist 
Admi nistrator 
Special Educator 
Regular Educator 



MR 


LD 


ED 


0 


47 


' T7 




25 


14 


T 

i 


^ 50 


18 


3 


40 


. 34 
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♦r ■ ^ Table 1[ 

Participation by Teanr' Members in Two Aspects of 
^ Teatn Oeclsioa-M^king Meetings^ 

■ , * 



Aspect 


Most Active 


Least Active 


Initiating Service 


' Principal . 


Parent * ■ 


Options 


LD Teacher 


Regular Teacher 




School Psychologist 




Initiating Goal s and 


.School Psychologist 


Principal 


Methods Statements 


LO Teacher 


Parent 






Regular Teacher 



4 



t 
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Table 8 

Ratings of Factors Influencing Outcomes of "f^a^ Meeti ngs 



Factor ■ , . ^ R^^i"9 

Mean SD 



Teacher reports of child's classroom achievement 


4 


40 


0 


.79 


Information from child's parent/guardians 


3 


70 


1 


.16 


Child's scores on achievement tests 


3 


68 


1 


.49 


Availability of s-ervices 


3 


64 


1 


.25 


Teacher reports of child's social behavio*^ 


3 


56 


1 


.18 


Observational data other than teacher reports 


3- 


49 


1 


.26 


Child ' s "scores on intelligence tests 


3 


23 


1 


.59 


^Ratings were on^a scale of 1 to 5/ where 1 = no 
very significant influence. 


i nfl uerice 


and 


-1 

5 - 


• 
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Table 9 

Perceatages of Meeting Time Spent 
Discussing Domaiq^s of Assessment Information 

. Type of Data \^ Percent Jime 

Classroom ^ • ^ 

Achievement ^ ' 14 
Intelligence * ^ 7 
Psychol inguisti'C ^ 4 
Perceptual -Motor ^""^ 2 
Personality ^ • - ' 2 
Othar non-test information _ 53 
^ V- 
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Table 10 

Correlations Between Eligibility Criteria 
and LD Classification Decisions by Placement Teams 



> 



Criterion 



Correlation 



Ability-Achievement Discrepancy 
Verbal-Performance Discrepancy 
, Federal Defi ni tion 'of Learning Disability 



.29 
.28 
-.1 3 
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PRETEST 



REFERRAL INFORMATION 



REVIEW OF ASSESSMENT DATA - TEST SCORES 

'TECHNICAL INFORMATION 
QUALITATIVE INFORMATION 



DECISION. QUESTIONS 



ELIGIBILITY 
CLASSIFICATION 
PROGf^OSIS 
PLACEMENT 

FACTORS AFFECTING DECISION 



EFFICACY 



Figure 1. Diagram of Process in the Simulati.on Study. 
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Figure 2. Distribution of Standard Scores on PIAT Math for LD anS Non-LD Subjects 
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Appendix B 



Suirmiary of Findings and Implications 

Note: The results presented represent a synthesis -of findings acrose studies utili- 
zing varying sample sizes, methodologies, etc. Therefore, readers ape en- 
couraged to examine the original sources^ which are listed in the 
bibliography. * * . ^ 

Research Findings 

1. A large number of assessment devices were selected and/or reported to be used by 
decision' makers; there was considerabl-e diversity among tl^ devices used, and many of 
the devices used were technically inadequate for the purpose at hand. 

2. Utilizing psychometric information, educators had *dif f iculty differentiating lo'w^ 
achievers from learning disabled students; students with an average psychometric pro- 
file were frequently classified as mentally retarded, learning disabled, and/or 
emotionally disturbed and said to be eligible for special education services. 

3. The referral statement is often a major determinant of the final decision reached. 

4. Most time in placement team meetings is spent in describing and discussing data 
about the student. These data are often derived from technically inadequate devices 
and often do not relate to the final decision made. 

5. Not all team members, particularly teachers an^ parents, actively participate in 
all aspects of the* decision-=-making process. ' r- • 

«♦ 

Implications for Training and Practice 

1. Do not ask "what tests do we g^ye?" Ask these questions: \ 

a. What decision are we trying to make? 

b. What behaviors must be sampled to make that decision? ^ . ^ 

c. Are the standardized measures U) be used technically adequate for the 
purpose at hand? v 

d. Have we integrated results from multiple assessment methods and multiple 
information sources before making a decision? "Are major findings<* consistent 
across data sources? 

e. ttaVe we employed a broad view of assessment including psychometric ^forma- 
tion, analysis of the learning environment, and "intervent ions th^t have 
been empl^^d? 

2. Have pre-referral interventions, such teacher-collected data, or documentation 
of teaching strategies employed in the classroom, b^en tried, discussed and evaluated 
prior to the deciS"ion to begin the assessment process? 

3. Does my team utilize criteria for effective team' decision making? Does my team 
need training in the team , decision-making process?^ ^ 

4. What school-based research would aid my team anfAor district in making more 
efficient and effective educatiq^l decisions? 



Yes No 
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CHAKACTERI 



EFFECTIVE TEAM MEETINGS 



l: 



5. 



7. 

8. 
9. 



Tln^ purpose of the meeting is clearly stated (either verbally 
or in a written agenda. 

Team members are informed that one of *lhe purposes of the meeting 
is cotfcpliance 'with due process legislation. 

Additional goals for the purpose of improving or evaluating 
team functioning or productivity are clearly stated. (An exaisple 
of these additional goals is: "Remember that we agreed to use ^ 
more ^^hayioral or pbserVational data in our decision making"). 

The roles of-^he team members are clearly ^defined ' (beyond name 
ar^ title)^. ^^^^ ^ . . ' > 

A ,3tatemeriJ^ ^^ isTli|to^ about the desirability of* participation 
by all d^am membCTs'., " . , ' 

The decision(s) t^S^? 
stated. ' ^ ' 



made during the meeting is(are) .clearly 



If more than 1 decision is to be made, they are identified as 
Separate. 



The reason for referral is clearly stated. 

A' member keep^ some written record of the meeting, 
filling out forms at the end of the meeting..) ' 



(Includes 



Data Presentation and Utilization 



10. -.Data are explained in terms of how they r^ate fo the problem 
(i.^., what they tell you, not just the scoje) 



11. 
12. 

.13. 
14. 



_^ 15. 

Team Process 



The student's strengths as well as weaknesses are discussed.. 

Comparisons occur 'across ^different sources of data (e^g. , class- 
room observation ^and standardized tests) with evaluation of 
implications. ' ' / ' 

Everyday behayioral and academic data abovit the (itiild are presented' 

Tfclp provision and modif icatidns which have been made in the 
regular classroom' in attempt to deal with the student*s 
prablem(s) are explained. * • 

Systematic behavioral observation data, as well as fdfrmal testing, 
are presented (i.e., a formal observati'on procedure was utilized. 



16. TeAmiHembers are attentive listeners (i".e., look at speaker, 
nod, etc.) . ' . ' 

17. Team members clarify others' remarks by questioning, para- 
phTasijg, or elaborating. < " ^ 

18. Team members ^seek information and opinions from others. 
19,, The team stays-^rn^ t3sk. • " - 



V 




■erlc 



1 1 
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Generating Alternatives ^ 
Yes No * 0 

, 20. For each' decision to be made the team produc-es a list of 

alternatives for the child's educational needs-. 

* ^ 21^ The team suspends evaluation of the alternatives until the 

list is completed.. 

Evaluating Alternatives • • 

22. The team states the criteria for evaluating the alternatives. 

23. ^ team member verbalizes the need to evaluate the placement 
decision on the basis of the least restrictive alternative. 




Each alternative is evaluatj^d in terms of the child's educa- 
tional needs or the selected criteria. 

Making the Fina^ Daglaj-on 



^25. Team members verbalize their opinions regarding the decision. 



26. 



Members attempt to reach thXougI>^discuss ion a decision ^^^^ 
all are willing to suppory (i.e^. , a consensus decision). 



27. A decision(s) is(are) mad^ 

28., The final decision is cle^ly sta'fc^d 
Implementing the Decision 

29. A* statement is made about the flexibility of the decision. 



36. A method for changing the decision is clearly stated. 

31. A method for evaluation of the decision is specified. 

32. A timetable for the prdgram is specified. 

33. The role of eacbtfteaia inember ir^ implementing the decision 

is fu'lly described. 

34. The team evaluates its meeting as having atCained/not attained 

^ its' goals for. the meeting. 

Meetings with Parents Present 

35. In the beginning of tiit^ meeting, the parents are asked about 

their expectations for- the mee'^ting^Ce .g. , the parents are 
asked what they hope to learn from the meeting). 

. 36. Th« parents are included through the use of direct questions 

to them, comments and explanations directed to them, and 
^ asking if they have questions. 

40 37^^*' The "parentsj input is requested during the meeting. 

38. The parents* input is .responded tojl^^araphrasing, comparing 

to other sources of information, etc. 

39. Language is at a level which the parents can understand. 

When technical terms are used, they are accurately defined^ 
in a wayb that parents can understand. 
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